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DETAILED ACTION 
Drawings 

1 . The drawings are objected to because figure 33. block 448, the word "welll" should be 
spelled —well-. A proposed drawing correction or corrected drawings are required in reply to the 
Office action to avoid abandonment of the application. The objection to the drawings will not be 
held in abeyance. 

Information Disclosure Statement 

2. The Examiner has considered the references cited in the IDS filed October 18, 2001 and 
an initialed copy of the IDS is attached to this office action. 

Claim Rejections - 35 USC § 102 

3. The following is a quotation of the appropriate paragraphs of 35 U.S. C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 

4. Claims 1,2, 5,6, 10, 13, 14, 17, 18, 22, 30, 31, 34, 35, and 39 are rejected under 35 
U.S.C. 102(b) as being clearly anticipated by DeTitta et al. (hereinafter DeTitta), US 6368402. 

As per claim 1, DeTitta teaches: 

A) at least one camera, (col. 10, 11. 41-42); 

B) an indexing device ("custom built travel image stand", col. 10, 11. 38-41 and Daedal X- 
Y Precision with 4 axis indexing card, col. 10, 11. 47-50) for sequentially placing said 
microscopic crystals in camera view of said at least one camera (col. 10, 11. 42-43); and 
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C) at least one computer (figure 9) programmed to control said indexing device and said 
at least one camera, wherein said at least one computer is programmed (col 10, 11. 63-65) to 
receive from said at least one camera images (col. 1 1, 11. 7-9) of said plurality of microscopic 
crystals, wherein said at least one computer is programmed to classify (encoded zero or one, col. 
1 1, 11. 14-23) said plurality of microscopic crystals. 

As per claim 2, DeTitta teaches: 

a computer monitor (col. 10, 1. 67-col. 1 1, 1. 3, figure 9 , element 36), wherein an operator 
interfacing with said at least one computer manually inputs a score (a "1" or "0", col. 14-17) to 
classify said plurality of microscopic crystals after observing said plurality of microscopic 
crystals on said computer monitor. 

As per claim 5, DeTitta teaches: 

wherein each of said plurality of microscopic crystals are contained within a 
drop of liquid (col. 9, 11. 52-57), wherein each drop of liquid is contained within a well of a 
micro-well plate (col. 10, 11. 2-3, see figures 1-4 ). 

As per claim 6, DeTitta teaches: 

wherein said plurality of microscopic crystals are a plurality of protein crystals inside the 
wells of at least one micro-well plate (col. 1 1, 11. 8-10, see figure 8). 
As per claim 10, DeTitta teaches: 

a frame grabber for receiving images of said plurality of microscopic crystals from said at 
least one camera (col. 10, 11. 52-53). 

As per claim 13, DeTitta teaches: 
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A) sequentially placing said plurality of microscopic crystals in camera- view (col. 10, 11. 
42-43) of at least one camera utilizing an indexing device (Daedal X-Y Precision with 4 axis 
indexing card, col 10, 11. 47-50); 

B) taking the images of said plurality of microscopic crystals with said at least one 
camera (video system records individual wells, col. 10, 11. 42-43 and feeds into a framegrabber, 
col 10, 11. 51-53), 

C) transferring said images to at least one computer (the images are recoded and saved to 
a file, col. 10, 11. 53-58), 

D) receiving said images at said at least one computer (col. 10, 1. 67-col. 1 1, 11. 3), and 

E) classifying said images at said at least one computer (col. 1 1, 11. 8-17), wherein said at 
least one computer is programmed to control said at least one indexing device and said at least 
one camera (the table motion and image capturing is computer controlled, col. 10, 11. 52-55). 

As per claim 14, DeTitta teaches: 

wherein said step of classifying said images at said at least one computer is done 
manually, wherein an operator interfacing with said at least one computer and viewing a 
computer monitor (col. 1 1, 11. 8-11) manually inputs into said at least one computer a score ("1" 
or "0", col. 1 1, 11. 14-15) to classify said plurality of microscopic crystals after observing said 
plurality of microscopic crystals on said computer monitor. 

As per claims 17, 18, and 22, these claims recite substantially the same limitations as 
claims 5, 6, and 10 above and analogous remarks apply. 

As per claims 30, 31, 34, 35, and 39, they recite substantially the same limitations as 
claims 1, 2, 5, 6, and 10, above and analogous remarks apply. 
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Claim Rejections - 35 USC § 103 

5. The following is a quotation of 35 U.S.C 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

6. Claims 3,15 and 32 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
DeTitta as applied to claiml, 13, and 30 above, and further in view of Jurisica, I et al. 
(hereinafter Jurisica), "Intelligent decision support for protein crystal growth, IBM Systems 
Journal, vol. 40, no. 2, 2001. 

As per claims 3,15 and 32, DeTitta teaches that the computer presents the views to a user 
to classify; DeTitta does not specifically teach automatic classification. However, Jurisica 
teaches: 

wherein said at least one computer automatically classifies said plurality of^microscopic 
crystals after receiving said images (p. 396, col 2, last 2 lines and over to p. 397, first paragraph 
and p. 402, section entitled 3. Classification). 

It would have been obvious to one of ordinary skill in the art to use the automatic 
classification system of Jurisica in the system of DeTitta to eliminate human intervention in 
order to cope with the high-throughput of the robotic setup (Jurisica, p. 398, col. 2, section 
entitled Image analysis). One of ordinary skill would be motivated to look at the Jurisica 
reference because DeTitta is a co-author of the article and DeTitta also uses a high-throughput 
robotic set-up. Use of the automatic classification feature would clearly increase system speed 
and efficiency once optimized. 
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As per claim 28, DeTitta does not specifically teach comparing images to images stored 
in the database, although he does teach comparing collected image data to data stored in the a 
database (comparison of precipitation scores, col 1 1, 11. 26-50). However, Jurisica teaches: 
step of classifying said images at said at least one computer comprises the sub steps of: 

A) comparing said images (figure 7, p. 403) to a database stored on said at least one computer (p. 
402, col. 1), and 

B) assigning a classification based on said comparison (uniform, granular or microcrystalline, p. 
402, col. 1), and 

As per claim 29, DeTitta teaches: 

A) making a determination as to whether a microscopic crystal exists (a "1", col. 11,1. 

15), 

B) making a determination as to what is present if said microscopic crystal does not exist, 
and (a "0", col. 1 1, 1. 14). However, DeTitta does not specifically teach a determination of 
quality although DeTitta does teach such a feature. 

C) making a determination as to the qualities of said microscopic crystal if said 
microscopic crystal does exist (uniform, granular or microcrystalline, p. 402, col. 1). 

Claims 4, 16, and 33 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
DeTitta, Harris and Update as applied to claims 7, 19, and 36 above, and further in view of 
Reimer et al., (hereinafter Reimer), US 60610861. 

As per claim 4, DeTitta does not specifically teach the use of two cameras, However, 
Reimer teaches: 
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wherein said at least one camera is a first camera and a second camera, wherein said first 
camera is set at a constant zoom to cover a large field of view (fig. 3, element 20, col. 6, 11. 61- 
63, wherein said second camera comprises a motorized zoom lens controllable (fig. 3, element 
30, col. 6, 11. 63-66) by said at least one computer (col. 7, 11 20-28). 

It would have been obvious to one of ordinary skill in the art to eliminate the data volume 
while still maintaining the processing capability equivalent to the human visual system, which 
provides the greatest accuracy in classifying image objects as defective (Reimer, col. 3, 11. 33- 
55). 

7. Claims 7, 19, and 36 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
DeTitta as applied to claims 1,13 and 30 above, and further in view of Harris et al., (hereinafter 
Harris), US63 88788 Bl and Automation Update (hereinafter Update), "Open for Inspection: 
Parker Daedal's New Positioner designed for Inspection Applications", Jan. 
1 998,www.textronics. com/ Newsletters/archivedarticles/1 998/0 1 98newsletter.pdf. 

As per claims 7, 19, and 36, DeTitta teaches: 

wherein said indexing device comprises: 

A. a first linear actuator for horizontally positioning said positioning station (Daedal X-Y 
Precision, col. 10, 11. 48-53), Update teaches that the X-Y Precision can accept microscopes and 
other machine vision inspection equipment. DeTitta nor Update specifically teach being able to 
move in a vertical axis, although DeTitta does teach that the plate glass and framework are 
controlled by a 4 axis indexing card. Four (4) axis to the examiner indicates movement in the X, 
Y, and Z axes. However Harris teaches: 



Application/Control Number: 09/982,048 Page 8 

Art Unit: 2623 

B. a second linear actuator for horizontally positioning said at least one camera (col 15, 
11. 3-5, although Harris teaches an XY controller moves the 96 well (microtiter) plate, it would 
have been obvious to one of ordinary skill in the art that the camera could be moved instead of 
the well plate), and 

C. a third linear actuator for vertically positioning said at least one camera (col. 14, 1. 66). 
It would have been obvious to one of ordinary skill in the art to use the suggestions of 

Update, which more thoroughly describes the custom capabilities of the Daedal X-Y Precision 
table disclosed in DeTitta, and those features disclosed in Harris to incorporate obvious camera 
manipulation apparatus into the custom built table and indexing device disclosed in DeTitta to 
provide the most cost effective by using commercial-of-the-shelf available parts and time 
efficient to keep up with the high throughput requirements of DeTitta (abstract) to build an 
indexing device. One of ordinary skill would be motivated to use the above prior art systems 
particularly since Harris is directed towards imaging a 96 well plate as is DeTitta. 
8. Claims 8, 20 and 37 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
DeTitta, Harris and Update as applied to claims 7, 19, and 36 above, and further in view of 
Reimer et al., (hereinafter Reimer), US 60610861. 

As per claims 8, 20 and 37, DeTitta nor Update teach using two cameras. Harris teaches 
using two cameras (col. 16, 11. 53-56), but does not teach one camera having a constant zoom 
and the other having a motorized zoom. Although Harris does teach a motorized zoom (col. 14, 
1. 66). However, Reimer teaches: 

wherein said at least one camera is a first camera and a second camera, wherein said first 
camera is set at a constant zoom to cover a large field of view (figure 3, element 20, col. 6, 11. 62- 
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63), wherein said second camera comprises a motorized zoom lens (col. 6, L 63-. col. 7, L 1) 
controllable by said at least one computer (col. 7, 11. 20-29). 

It would have been obvious to one of ordinary skill in the art to use the two camera 
feature of Reimer in the system of DeTitta, Update and Harris, to provide accuracy in finding the ' 
well locations by allowing the wide field of view camera image to be scanned to locate target 
(wells) images, particularly since both DeTitta and Harris are directed towards locating well 
positions and performing high speed classification. 

9. Claims 9, 1 1, 12, 21, 23, 24, 38, 40, and 41 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over DeTitta as applied to claims 1, 13, and 30 above, and further in view of 
Inokuchi, US 6259960 Bl. 

As per claims 9, 21, and 38, DeTitta discloses that the user can name a plate manually, 
but does not specifically state that a barcode is used. Nor does DeTitta teach a plate sensing 
device. However, Inokuchi teaches: 

comprising a bar-code reader (col. 7, 11. 36-40 and col 18, 11. 39-41) and a micro-well 
plate sensing device (part moving means moves the part to be inspected into the target coordinate 
region, col. 7, 11. 60-65, or alternatively, the state coordinate system is adjusted based upon the 
imaged x-y difference between the target coordinates and the state coordinates, col. 8, 11. 23-39). 

It would have been obvious to one of ordinary skill in the art to use Inokuchi's automated 
bar-code reader to input the information regarding the plate instead of manually inputting that 
data into the computer which may result in mistaken information. Also, Inokuchi teaches that it 
is well known to have position sensing capabilities in inspection devices, which facilitates 
automated inspection because precise placement of the target plate is not a true concern. One of 
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ordinary skill in the art would have been motivated to look at the inspection device of Inokuchi 
because it is directed towards automatic image processing of microscopic images for certain 
features to be found in the image. Inokuchi also has a manual inspection mode as well as an 
automatic inspection mode. 

As per claims 1 1, 23, and 40, although DeTitta teaches that his inspection system is used 
on a computer, he does not specifically teach that is part of a network. Inokuchi is directed to a 
material- inspection and classification apparatus (col. 17, 11. 18-21, 24-26). However, as per 
claims 1 1, 23, and 40, Inokuchi teaches: 

wherein said at least one computer is connected to a computer network, (figure 1, col. 16, 
11. 60-65). 

It would have been obvious to one of ordinary skill in the art to use the networking 
suggestion of Inokuchi in the inspection and classification system of DeTitta to facilitate 
management of classification information at a central location thereby making it unnecessary to 
save and download classification data from individual plate imaging stations of DeTitta 
(Inokuchi, col. 17, 11. 43-50). 

As per claims 12, 24, and 41, Inokuchi teaches: 

wherein said at least one computer network is the Ethernet, (col. 16, 1. 65). 
10. Claims 25 and 26 are rejected under 35 U.S.C. 103(a) as being unpatentable over DeTitta 
as applied to claims 13 above, and further in view of Bacus et al., (hereinafter Bacus), US 
6226392 Bl. 

As per claim 25, DeTitta does not specifically teach color images are obtained of the 
crystals. However, Bacus teaches: 
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wherein said images of said plurality of microscopic crystals are color images, (col. 8, 11. 

56- 58) 

It would have been obvious to one of ordinary skill in the art to use the color filter wheel 
of Bacus with the imaging sensor of DeTitta to provide color images that will allow for more 
accurate classification of crystals when classification may be based upon the certain colors in the 
crystals. Furthermore, Bacus teaches that color images have been well known in inspection 
systems for a number of years. 

As per claim 26, Bacus teaches: 

A) viewing each of said plurality of microscopic crystals through a first linear polarized 
filter (figure 4B, elements 172 or 170) and a second colored filter (figure 4B, element 180), 
wherein the color of said second colored filter is altered between red, green and blue (col. 8, 11. 

57- 58), to obtain a first image, a second image and a third image, and 

B) combining said first image, said second image and said third image to obtain said true colored 
images (col. 8, 11. 59-62). 

1 1 . Claim 27 is rejected under 35 U.S.C. 103(a) as being unpatentable over DeTitta and 
Bacus as applied to claims 13 above, and further in view of Glazer et al (hereinafter Glazer), UK 
2310925. 

As per claim 27, Bacus does not specifically teach using false colors. However, Galzer 
teaches: 

steps of viewing each of said plurality of microscopic crystals through a first linear 
polarized filter (figure 1, element 15) and a second linear polarized filter (figure 1, element 14) 
to obtain said false color images (p. 2, second paragraph). 



• 
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It would have been obvious to one of ordinary skill in the art to use the false colors of 
Glazer in the color crystal images of DeTitta and Bacus to measure the refractive indices of a 
crystal which by using false colors allow for indications or measures across the sample for its 
optical anisotropy, which provides additional data to further classify any crystal material and 
take advantage of the image processing inspection system. 



12. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. The following U.S. patent(s) refer(s) to color filtering: Bos et al., 5714970, Miller, 
5689317, and Treado, 6002476. 

13. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Martin Miller whose telephone number is (703) 306-9134. The 
examiner can normally be reached on Monday-Friday, Maxi-flex. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Amelia Au can be reached on (703) 308-6604. The fax phone numbers for the 
organization where this application or proceeding is assigned are (703) 872-93 14 for regular 
communications and (703) 872-9314 for After Final communications. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the receptionist whose telephone number is (703) 305-3900. 

/ M 

mem / yy// 

August 8, 2002 L^ff /; : ! * 
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